Correctness of Local Probability Propagation in GraphicalModels

Graphical models, such as Bayesian networks and Markov networks, represent joint distributions over a set of variables by means of a graph. When the graph is singly connected, local propagation rules of the sort proposed by Pearl (1988) are guaranteed to converge to the correct posterior probabilities. Recently, a number of researchers have empirically demonstrated good performance of these same local propagation schemes on graphs with loops, but a theoretical understanding of this performance has yet to be achieved. For graphical models with a single loop, we derive an analytical relationship between the probabilities computed using local propagation and the correct marginals. Using this relationship we show a category of graphical models with loops for which local propagation gives rise to provably optimal MAP assignments (although the computed marginals will be incorrect). We also show how nodes can use local information in the messages they receive in order to correct their computed marginals. We discuss how these results can be extended to graphical models with multiple loops and show simulation results suggesting that some properties of propagation on single loop graphs may hold for a larger class of graphs. Speciically we discuss the implication of our results for understanding a class of recently proposed error-correcting codes known as \Turbo codes". A B C D a b c Figure 1: Examples of graphical models. Nodes represent variables and the qualitative aspects of the joint probability function are represented by properties of the graph. Shaded nodes represent observed variables. a. A singly connected Bayesian network. b. A singly connected Markov network (A and F are observed). c. A Markov network with a loop. This paper analyzes the behavior of local propagation rules in graphical models with a loop.